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Abstract — Demodulation in a random multiple access channel 
is considered where the signals are chosen uniformly randomly 
with unit energy, a model applicable to several modern trans- 
mission systems. It is shown that by lifting (replicating) the 
graph of this system and randomizing the graph connections, 
a simple iterative cancellation demodulator can be constructed 
which achieves the same performance as an optimal symbol-by- 
symbol detector of the original system. The iterative detector has 
a complexity that is linear in the number of users, while the direct 
optimal approach is known to be NP-hard. However, the maximal 
system load of this lifted graph is limited to a < 2.07, even for 
signal-to-noise ratios going to infinity - the system is interference 
limited. We then show that by introducing spatial coupling 
between subsequent lifted graphs, and anchoring the initial 
graphs, this limitation can be avoided and arbitrary system loads 
are achievable. Our results apply to several well-documented 
system proposals, such as IDMA, partitioned spreading, and 
certain forms of MIMO communications. 
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I. Introduction 

ITERATIVE graph-based signal processing has enjoyed a 
tremendous rise in popularity with the introduction of 
turbo coding Q. By breaking complex algorithms into local 
processes which communicate in iterative cycles, many highly 
complex problems can now be addressed with relative ease, 
and the iterative process reduces complexity in a way similar 
to how iterative solution methods break down the problem 
of solving large systems of equations into a repetition of 
operations with much lower complexity |21 J. 

The analysis of the performance of iterative processors, on 
the other hand, has offered more resistance, and relatively little 
is known about the exact performance of these algorithms. 
The methods introduced by ten Brink |30|, Divsalar [6|, and 
Richardson et. al. ||19J are statistical methods to predict the 
average dynamical behavior of large iterative error control 
decoders. These analysis methods have much in common with 
Gallager's original probability propagation approach used to 
study low-density parity-check codes 121. The same type of 
analysis has been utilized by others fl\, 1251 . ||3] to study 
the behavior of iterative decoders for multiple access systems, 
which were introduced shortly after the invention of the turbo 
decoding algorithm ifTSl . Ill7l . [l]. All of these methods can 
predict the dynamical behavior of iterative processors for 
large-scale systems with remarkable accuracy, however, they 
cannot predict exact performance in terms of error rates, etc. 
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The question how close iterative processors, in particular 
decoders and demodulators, can approach the performance 
of an optimal decoder is also not well understood. While it 
appears that the turbo decoding algorithm has a performance 
very close to that of a maximum-likelihood processor, it is also 
know that it is not equivalent to maximum likelihood llT4l . 

In this paper we present a case where an augmented graph- 
based iterative processor achieves the optimal maximum- 
likelihood performance of the underlying original system in 
terms of error rates (and LLR statistics). The system in 
question is a multiple access system where the waveforms 
used for modulation are uniformly randomly selected from the 
A^— 1-dimensional unit(-energy) sphere in A^-dimension. This 
model has practical counterparts in random CDMA (code- 
division multiple-access) as explored in partitioned CDMA 
fST], fSSl, interleave-division multiple access (IDMA) |16|, or 
in isotropic multiple antenna communications channel models. 

The performance of a maximum-likelihood detector for 
random CDMA was calculated in f281 where a statistical 
mechanics approach was used. In this paper we show that 
a proper augmentation of the factor graph of the random-code 
model leads to a system in which a basic message passing 
algorithm can be applied which achieves the same performance 
as the optimal detector in the original system as computed 
inlH. 

However, from a performance standpoint, the iterative sys- 
tem is ultimately interference limited in the sense that the 
maximal ratio of signals per dimensions a = K/N < 2.07 
in an equal received power situation. We then show that this 
interference limitation can be overcome by a process called 
spatial coupling, whereby successive frames of the original 
system are interconnected, and the first few frames are initiated 
with known symbols which act as anchor or pilot symbols. 
We show that with appropriate such anchoring the interference 
limitation disappears and the supportable loads a — > oo, given 
sufficient signal-to-noise ratio. 



II. System Model 

We consider a general communications system with K data 
streams. Without essential loss of generality, the data consists 
of independent binary symbols dk G {—1, 1}, k — 1, . . . ,K 
which may be the outputs of K parallel forward error (EEC) 
control encoders. The binary symbols are then modulated onto 
random unit-energy waveforms aj. chosen uniformly from the 
iV-4 dimensional unit sphere in N dimensions - see Appendix 
II for the signal description. 
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The composite signal after transmission at the receiver is 
given by 



K 



dk&k 



an 



(1) 



fc=i 

where n is a vector of unit energy noise samples, and cr is 
the noise standard deviation. As shown in Appendix II, the 
average cross-correlation of these waveforms is E[ai[aj] = 
l/N, k ^ j, which is all we neeqM 

In the sequel we will use graph-based arguments and we 
therefore introduce the "factor graph" IfTTl . |[10| of (fTl), shown 
in Figure [T] where the center node symbolizes the addition of 
the K signals. 

{^2, ■ ■ • ,dM~l} 



IK 




Fig. 1. Factor graph of the basic multiple-access system of Equation [T] 

A decoder, or demodulator, can utilize this graph as an 
algorithmic blueprint. In this case the graph is not very 
interesting, and consists of a single, cycle-free star graph. An 
(optimal) demodulator will have to compute 



d = argmin ||y 

d 



K 

E 

k = l 



rffcafcl 



(2) 



which involves the computation of 2^ terms. In any case, it 
is known that (pi) is NP-hard ll34ll . and there is little hope that 
efficient algorithms for its exact solution exist unless P=NP, 
which most experts agree is not likely the case [8J. 

Even though the general problem is NP-complete, special 
instances of the multiple access problem (|2]) have efficient, 
low-complexity solution algorithms. For example, if a^a; = 



0, / ^ fc, which is usually done by design, see e.g. 
demodulation complexity is of order 0{NK) only. If the 
cross-correlations between the different signals take on only 
nonpositive values, optimal detection corresponds to a solu- 
tion of a min-cut problem of the associated graph, whose 
complexity is 0{NK^) 1201 . Il32i . Sequences a^ can also 
be constructed specifically such that lower-complexity optimal 
detection is possible, even when the system is over-saturated, 
i.e., a = K/N > 1 1131, where a is called the system load. 
In ll24l the authors introduce controlled additional sequences 
such that the corresponding over-saturated system can be 
demodulated with a trellis decoder, whose complexity depends 
on a. All these approaches, of course, are only feasible if 
timing of the different signals can be tightly controlled and 
no transmission distortions occur. 

Like the optimal sequence detector, the symbol-wise 
marginal-posterior-mode (MPM) detector, which maximizes 
the symbol-wise posteriori probability, is NP-hard in general. 
The probability of error for the MPM detector was com- 
puted in 1281 using statistical mechanics tools. The signal-to- 
interference ratio 7^ of the posterior probabilities of d^ prior 

'Note that the waveforms can. of course, be complex as is the case in 
typical wireless communications systems. 



to thresholding is computed as the solution to the implicit 
equation (IH Eqns. 45 and 43]) 

-1 



7 



a^ + aE 



1 - tanh ( ^ (7' + 70 



(3) 



where ^ ~ A/'(0, 1), and E and F are given in |28, Eqn. 43], 
and are the mean and variance of the (individual) APP output 
signal. In general E > F, however in the case of binary 
random signaling as used in this paper E — F, which follows 
from the definitions of Tn,q,E,F in Ii28i Eqn. 27 ff.]. The 
result ^ holds for loads a < a^ « 1.49, where as is the 
"spinodal value", where the statistical mechanics approach 
undergoes a phase transition and multiple solutions appear 

Mathematically, the spinodal value separates the region 
where a single solution exists to ([3]l, from that where multiple 
solutions exist. More precisely. 



aE 



1 — tanh 
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is monotonically non-decreasing for in x a < as, and 
therefore only a single solution to (J3]l exists. For a > as there 
are multiple solutions, and the iterative detector presented here 
will converge to the solution with minimum 7. 

In this paper we show that by process called graph lifting 
and randomization of connection, we can obtain an instance 
of the decoding problem where an iterative message passing 
algorithm can achieve a posterior LLR signal-to-interference 
ratio that is identical to ([3]), and therefore achieves the same 
error performance as the MPM detector, but with a decoding 
complexity of 0{IN) per bit dk, where / is the number of 
iterations. The number of iterations / does not depend on 
K, and in most cases, especially those of practical interest, 
0{IN) < 0{N2^/K). 

III. Graph Lifting 

We now consider an equivalent model for the system (fill. 
Imagine that each bit is replilcated M times as dk,m — 
dk,yrn = 1, • • • , M, i.e., the transmission model is 



KM, 

E\~^ dk,m 

fc — 1 771—1 

The factor graph of (J5]) is shown in Figure [2] 

di.i dx.i 



(5) 




K,M 



Fig. 2. Factor graph of the basic multiple-access system of Equation [T] 
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However, since Q is linear, we find the representation in 
Figure [5] more usefiil. Note that the central multiple access 
node now has an iV/-fold larger incidence degree. 




K,M 



Fig. 3. Factor graph of the basic multiple-access system of EquationfT] 

Figure HI shows an L-fold lifting of the graph of Figure [3] 
that is, basically an L-fold repetition. This can be thought of in 
practice as the transmission of a block of LM signals instead 
of the original AI signals for each user. 




K M 



^k,m' ,1' 



k=lm=l ^ 



■SLk^i' + ani 



(6) 



where we have introduced the graph copy indices I and I', and 
(m', I') are simply the indices of the original location All'+m' 
of an edge that is permuted to location Ml + to in (jSl. As a 
small detail we note that in order to keep the total energy of 
the system constant, we had to change the amplitudes of the 
waveforms to 1/\/a7. This is because the individual portions 
of a given waveform do no longer add coherently as in (J5]l. 
Figure E\ shows the resulting graphical model. 




(to,0 



{m,l) 



Fig. 5. Graphical structure created from the L-fold lifting of the original 
factor graph by adding individual permuters. 




Fig. 4. L-fold lifting of the factor graph of the basic multiple-access system 
of Equation [T] 



Of course, up to now nothing has changed in the system 
or the processing of the data. However, analogously to the 
construction of LDPC codes from protograph structures ||3T| , 
Q, a concept initiated by Tanner ||29l , we now permute 
the edges among the different copies of the graph lifting. 
This can be done in a number of convenient ways, such as 
to avoid contention by a memory-based receiver processor 
||271 . However, here we simply assume that the permutations 
are done randomly, and independently from each other. The 
resulting multiple access equation is given as 



The key observation is, that instead of solving the NP- 
hard decoding problem for ([T]i, we apply an iterative message 
passing algorithm to the lifted graph in Figure E\ and we show 
that this iterative algorithm achieves the same performance as 
the maximum-likelihood algorithm for the original problem. 
This is significant in that the exponential complexity per bit 
has been transformed into the complexity of the message 
passing algorithm per graph section. The latter, as we will 
show, only grows linearly in K. 

IV. BELIEF Propagation Message Passing 

Message passing in the lifted graph is carried out in the 
usual way. The equality nodes compute an extrinsic log- 
likelihood ratio of the bits dkjn,i as the sum of the incoming 
a priori log-likelihood ratios, i.e.. 



^{dk,m,,l) — 2^ ^{dk,n,l) 



(7) 



7i:7i^7n 
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where \{d) = log (Pr((i = 1)/Pr((i = — 1)) formally plays 
the role of a log-likelihood ratio. The details of the message 
passing operation are illustrated in Figure l6] 
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Fig. 6. Illustration of the message exchange mechanism and node function- 
alities in the message passing algorithm. 

Message passing through the multiple access node is a little 
more involved, and an exact calculation of the extrinsic log- 
likelihood ratio would lead us back to the NP-hard problem 
we wanted to avoid in the first place. We therefore utilize a 
much simpler cancelation and subsequent unconstrained LLR 
computation. For each outgoing edge from the I'-ih multiple 
access node, the following cancelled and filtered signal is 
computed and sent to the symbol nodes: 



^k,rrt'l' 



X{dk,m',v) — 



^k,V 



Yl' 






tanh 



KdK.,^l.v)\ a^u 



Vm 



(K,f<)^(fc.m') 



Ma- 



' ^k,rfi' ,1' 



(8) 



where af is the joint multiple access and noise interference 
of the (partially) canceled signal Zf^i^^^iiiy In the limit of 
K -^ oo, by virtue of being a sum of i.i.d. bounded random 
variables Zk^m',i') becomes Gaussian with variance af and 
mean dk,{rn'.i')/V^ by the central limit theorem. Evidently, 
its variance is independent of the lifting index /, the user index 
k, and the partition index m. The index i can be understood as 
an iteration index, since this variance changes with repeated 
iterations of the processing operations (J7]| and (IHll - see also 
Figure |6] 



For the signals used E 



a,, ,&„ 



= 1/N, (see Ap- 
are distributed 



pendix II), and noting that the LLRs in 

with mean 2/{Maf) and variance A/{Maf), and accounting 

for 0, it follows that 

H 2 
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(9) 



where ^ ^ A/'(0, 1). That is, assuming that (J8| describes a 
Gaussian random variable - which is true for K large - (J9| 
is an exact expression for all i < /cycio/2, where /cycle is the 
length of the shortest cycle in the graph of Figure l5] Equation 
(|9]l is therefore exact for as long as the system size L is large 
enough to ensure large minimum cycles. Typically this means 
that L grows as ex e^"^''^". Nonetheless, the analysis doesn't 
collapse for shorter L, but becomes increasingly approximate. 
We shall assume in what follows that i — > oo, without 
deriving any practical suggestions from this statement. 

Equation (J9]l is monotonically decreasing due to the mono- 
tonicity of the expectation function in (|9]l. It is a contraction 
map with a unique fixed point according to the Banach fixed 
point theorem. This fixed point is given implicitly by 
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I M-l 
Mal^ 



(10) 



where in the second row we have used KM S> 1, which is 
the case for most situations of practical interest. 

Note that the analysis up to this point does not depend on 
the specific binary sequence chosen in the K data streams, and 
one can assume the all-zero sequence for all of them, also that 
is not required. 

At each iteration, the equality node will collect the LLR 
messages reported from the network and form an i-th soft 
symbol estimate (or a posteriori LLR value) to be reported to 
the external system. This LLR is given by 



M 



Kdk.i) = E '^('^'= 



(11) 



for each binary symbol dk.i- Following (|6|, ( |T0| ), and (J7|, the 
signal-to-noise ratio of this estimate is given by ^f — l/af, 
and follows the iteration equation 



aE 



1 — tanh 



M-l 



'-It- 1 + 



M-l 
~Af 



-li-ii 



(12) 



We note that for large partition numbers il/ — ^ oo, the 



fixed point SNR of ( 12 1 and the implicit SNR equation for the 
optimal symbol detector (Bj) are identical. That is, the SNR ^f 
approaches the optimal symbol SNR as iterations approach 
the fixed point. This shows that simple message passing in 
the (large) lifted and randomized graph of the random signal 
multiple access system achieves optimal performance, i.e., its 
error performance is identical to that of operating an NP-hard 
maximum marginal probability detector on the original system 
(also the detected binary sequences would not necessarily be 
identical with probability one for any finite signal-to-noise 
ratio). 
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V. Discussion 



For cr2 > 0-2^.^ 



0.148, equation (12i has only a single 
solution. However, for cP' < cr^^j^, one, two or three solutions 
exist, depending both on a^ as well as a. The relevant solution, 
i.e., the fixed point approached by the iterative detector, is the 
one with the largest al^. Only the case with a single solution 
is practically relevant, since the other solutions are high-noise 
fix points where the decoder essentially fails, i.e., produces 
high bit error rates (see Figure M below). For each a"^ < cr'^j.^^, 
we have an associated maximum a such that a single solution 
exists. For <j^ — > 0, this maximum a is the total maximum 
system load that is supportable at any SNR, given by a,nax = 
2.07425. 

Figure 17] shows how the appearance of additional solutions 
translates into a "turbo-cliff behavior of the bit error rate. 
This abrupt phase change is therefore characteristic of the 
optimal detector as well. Our results also suggest that the phase 
transitions observed in the statistical mechanics approach ll28l 
Figure 1] are artifacts of the analysis, and are not a phe- 
nomenon of the detector itself, whose error curves transition 
sharply as the fixed point jumps at the critical SNR. 

The bit error rate was computed from the signal-to-noise 
ratio 7^ as Pf, = Q(7oo)- The dotted line is the bit error rate 
of antipodal signaling on an AWGN channel. We can deduce, 
for example, that at a = 1 the cost of using random signaling 
instead of orthogonal signals is about 2dB at Pi, — 10^^, and 
shrinks to a small fraction of a dB for lower error rates. 
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Fig. 7. Bit error rate of the MPM detector for various a = K/N. 



Just as with irregular low-density parity-check codes, where 
an optimization of the node degrees led to significant im- 
provements in performance ITSl . the partition degree of our 
lifted graph does not have to be equal for all users, or all 
symbols for that matter We are free to chose a partition degree 
profile Afi 1, • • • , Mi^K, ^^2.1, • • • > M^^k- The fact that gains 
in performance can be achieved can be seen through the 
following argument. Letting Mi,k = Mk, it is not hard to 
see that, as long as each symbol is transmitted with constant 
power 1/Af, this amounts to assigning different powers to 
the different users. However, it is already known that with 
exponentially distributed power assignments, or equivalent 



a3,3 -<-' 




ai,i 



as,! 



a^ 



33,1 



Fig. 8. Illustration of the redundant signal population using the lifted system. 

rate assignments, the multiple access channel capacity can be 
achieved with a cancelation system of this type |23|. 

Lastly, we wish to comment on a subtle difference in the 
use of the signal space of the lifted system, compared to the 
original system. As mentioned in Section II (and Appendix 11), 
the signals a are drawn uniformly and randomly from the A^- 
dimensional signal space. With the lifting, each signal is split 
into M signals, also drawn uniformly randomly. Note that the 
total power and the total bandwidth utilization are equal in 
both systems. However, the lifted signal has a higher signal 
density, as M times as many signal crowd the original signal 
space. This is illustrated in Figure [8] 

Note that without loss generality to the analysis we may 
wish to enforce that the signals belonging to the same user 
k, and the same time interval I, are chosen to be mutually 
orthogonal, i.e., afc,r,i -L SLk^v^2 J- • • ■ J- SLk,i'M- This causes 
no significant design overhead, since instead of using M 
random signals, each user now uses a random M-dimensional 
basis to transmit its symbol fragments in time interval /. 
The analysis presented in this paper carries through without 
change. However, from an operational point of view, signal 
processing at the receiver is facilitated since no intra-user inter- 
signal interference needs to be cancelled. 

VI. Spatial Coupling 

In this section we are addressing the issue of the maximum 
system load of a„iax introduced in Section [V] The question is 
in what way this is a limit to the iterative detector - in short, 
it is not. This will be demonstrated in this section. 

In the development of LDPC convolutional coding Q, a 
phenomenon called "spatial coupling" has allowed these codes 
to be designed with decoding thresholds that are very close 
to the channel capacity. The effect of spatial coupling derives 
from anchoring initial symbols to known values, which causes 
a locally smaller rate. This in turn allows the code to converge 
in signal-to-noise ratios where uniform convergence is not 
possible. Recently, it has been shown that spatial coupling 
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can decrease the convergence threshold in low-density parity- 
check codes on binary-erasure channels all the way to the 
maximum-likelihood threshold |12|. 

In our system we introduce spatial coupling by duplicating 
the graphical structures of FigurelS] i.e., lifting the graph again. 
One can think of this as repeating signaling in time. We then 
anchor the signals of the left-most graph to known values, 
these can be thought of as pilot signals, for example. The 
spatial coupling in its simplest form is achieved by connection 
of a fraction a of the nodes of the graph at lifting level (time) 
t, to the multiple access nodes at time t—1. This is illustrated 
in Figure l9] where the solid black nodes denote the anchored 
known symbols. 

By this process, the first set of multiple access nodes effec- 
tively experiences a lower load, and will therefore converge 
faster. Astoundingly, this effect propagates through the semi- 
infinite lifted graph indefinitely. 

In order to simplify the notation and following discussion 
we define 



I'M - 1 
I Mx 



= E 



1 — tanh 




(13) 



and we note that g { ^fj^ ) is a monotonically increasing 
function in a; lH. We also assume without essential loss of 
generality that M is large, so we can approximate M— 1 w M. 
For the further discussion we substitute x* = cr^j. A density 
evolution analysis for large graphs then links the variances at 
the different times via the following equation. 

1 /I — a a \ o 

X- = aag -^ — + -j — + a^ (14) 



aag\ 



r.t + 1 



-a{l- 



The question now is if the iterative system of equations ([14]) 
allows convergence at higher values of a > Ofmax- The answer 
is affirmative, however the maximum load a now depends 
on the coupling connections, but ttcoupiing > ctmax for any 
coupling. In particluar, for the coupling of Figure l9] with a = 
0.5, acoupiing = 2.81. 

Figure 10 shows that the variance values at the subgraphs at 
different indexes t evolve completely homogeneously, with a 
constant iteration delay towards larger t. That is, the solutions 
to the balance equations ^T4\ are linear shifts of each other. 

Figure 1 1 shows the convergence of the variance values at 
the subgraphs at for an signal-to-noise ratio of lOdB. At that 
noise level, the system convergence is shown for a = 1.95, 
which is significantly larger than without spatial coupling - 
see Figure It] 

We may generalize the coupling to include W previous and 
W future graphs, i.e., utilize a coupling window of 2W + 1, 



instead of of the basic example in ( 14 1. The first W subgraphs 



are now the anchor symbols and are set to fixed values. The 
variance update equations at time t can now be computed as 
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1 



2W 



1 ^^ 

j=-W 
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^ 2W + 1 a;*+J+' 

kI=-W ■^i-l , 



(15) 
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Fig. 10. Homogeneous convergence in the simply coupled graph of Figurel9] 
with a = 0.5 and a = 2.8 in the noiseless case. 




300 400 

iterations i 

Fig. 11. Convergence in the simply coupled graph of Figure l9] with a = 0.5 
and a = 1.95 in and SNR=10dB, hence the floor at x* = 0.1. 



This allows us to achieve progressively larger maximum values 
^coupling where convergence can be achieved, as substantiated 
by 

Theorem 1: For a^ = 0, and with a sufficiently large W, 
i.e., W = O (e"'"^"), the iteration equations ([15b converge to 
xl -^ 0,Vi and for all values of a. In particular we can let 

a -^ cx). 



Proof: See Appendix I 

Theorem [T] shows that the spatially coupled iterative de- 
modulator is no longer "interference" limited, in the sense 
that it can support an arbitrarily large system load given the 
necessary signal-to-noise ratio. This is in marked contrast to 
all known equal-power multi-user detectors, in particular also 
to the detector in Figure ]5] Some values of achievable loads 
are shown in the table below. 
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Fig. 9. Basic graphical coupling of different or successive lifted grapfis. Couplings with wider span will be more effective. 



TABLE I 

Achievable loads as a function of coupling window size. 



w 





1 


2 


3 


4 


5 


10 


20 


50 


•^coupling 


2.07 


3.17 


3.6 


3.9 


4.1 


4.3 


4.9 


5.5 


6.2 



Appendix A 
Proof of Theorem[T] 

In the spatially coupled system of ( 15 1 let y* denote the soft 



bit variance in the graph replica t. Then 

w 



a 



x', = ^,^ ^ y'+^ t>0,^>0 

3=-W 



2W 
and the soft bit variance 



2/j = S 



\2W + 1 ^ x*+\ I 



t>OA>0 



(16) 



(17) 



Combining these two equations we obtain 

w 



2W + 1 .^,/' 

j=-W 



2W 



is increased by a/{2W + 1). As a result, the initial condition 
can be formulated as 



x* = t<0 , 

t at 



2W + 1 



cr^; te[l,2W + l] 



and 



xi = a + (j'' t>2W + 1 



2/0-0 t<W 



2/0 = 1 t>W + l 



(19) 
(20) 
(21) 



(22) 
(23) 



We restate Theorem [T] in a slightly more detailed way for 
the purpose of this proof. 

Theorem 1: Consider an interference limited system with 
CT^ = 0, a > 2 and 



W > exp(15a In a + 20a) 



(24) 



Then for any t 



lim Xj 



where after i >> 2i iterations x* converges to at least 



(18) 



exponentially. (The specific numbers 15 and 20 in (24i are 
merely for convenience). 

The proof is broken into two parts. First we show in 
Lemma [T] that for W satisfying (24i and after 2t iterations 



We assume that transmission starts at time t = 1, i.e., the load 
of the system increases gradually. At every replica t, the rate 



Trexp 



1 

AaiT 



(2W^ + l)i5 



(25) 
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In the next step (Lemma |2]) we prove that 

for i > 2t + 2W. The effect of spatial coupling initiated at 
position requires 2t iterations to propagate to position t. 

Lemma 1: Assume that for t' < t we have noise and 
interference variance a;*_i < ea and for t' < t + W we have 



and therefore 



,,t+w 



/ln(2W^ + l 

- n 



< ttQ 



< TT exp 



,t' 



2a 

H2W + 1) ^ 
2a , 

ln(2Ty + l) 



Aa 



soft bit variance yl_-^ < e were e is given by (25 1. Then 

.,t+w 



< TT{2w + iy^ 



and as a result a;*^2 



yi+2 <e 
< ae. 



Combining ( 26 1 and d28b we obtain 



X, < ea 



-TT{2W + l)-r. 



Note that the initial conditions for spatially coupled system 
for i = 1 are satisfying the conditions of the lemma. Once 
the lemma is applied to replica t is can be repeated again for 
replica i + 1 and so on. 

Proof: It is easy to see that 

vLi < 1 fort' >t + W 



2W + 1 
Likewise for W satisfying (J24]i, and again using ( 25 1 

hence 



xl < 2an{2W +1) 



-i-i 



Now 



According to U6[, the above equation, and the condition of 
the theorem, (namely y*_ ^ < e for t' < t + W) we obtain 



yltV < 9 



< 



1 



1 



2W + 1 2aT:{2W + ly^-r. 
1 



w 



t+j 



t+j+i 



2W 



T E yl^i 



< Trexp 



2a7r(2M^ + l)-4 

l{2W+l)r. 



i=-w 



2aTr 



< 



2W + 1 



{{2W - j)e + j + 1); jG[0, 2W] (26) therefore 



Let us proceed with iteration i. According to (17 1 and the 
equation above we obtain 



X* , 1 < e{2W + l)a/{2W + 1) = ae 



.,t+w 




Let Edit be the smallest positive root of 

1 



exp 
Notice that for e < e^rit 
exp 



2ae 



2ae 



6/2 



<e/2 



1 1 

2Wt + l ^ (2VK-l)e + 2 



(28) 



(29) 



(30) 



(31) 



and therefore 

,,t+w 



Vr 



<9 



1 



1 



1 



^a2We+l 
since for any fc > integer 

2We+l 



{2W-k)e 



+ . 



1 
— > - 

k - k 



1 
21^ + 1 



1 
2W + 1 



moreover, e defined by (25 i satisfies (31 1. 

Lemma 2: Assume that for any t we have x\_i < ea and 
yj-i <e.lfe< Ecrit then 

yl < e/2 

and as a result x* < ae/2 for any t as well. 
Since e defined by ( [25] l satisfies ( [3T| i 

for i > 2t + 2W. This means that for T >> t, and after i > T 
iterations x* will decrease to zero exponentially. 



Proof 



For W satisfying ( p4| ), and e as in ( 25 1, it can be shown 
that 



< 



1 



1 



2T4^ + 1 V ae 
1 



1 
ae 



< exp(- — X./2 



(32) 
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Appendix B 
Random Signal Waveforms 

We are using a set of random unit-length vectors in ri-space. 
By random we mean that the vectors are distributed such that 
their endpoints form a uniform distribution on the n — 1- 
dimensional sphere that bounds the n-sphere within which 
these vectors live. 

Prelimaries: 

Consider two vectors x and y in 2 dimensions. Then the 
interference power is given by their crossproduct, i.e., 

p2 = (x.y)2 = |xny|2cos2(0) 

where (j) is the angle between them. 

Averaging this over (j> e [0, tt] gives 0.5. 

Lemma 3: The projection power of two random vectors x 
and y in the sense above is given by 

1 



P^ = (x • y)2 = 



(33) 



Proof: Assume without loss of generality that the first basis 
vector of n-space coincides with x. Now decompose x into 
two orthogonal parts, Xp — xcos(0) and x^ = xsin(0). Given 
that the endpoint of x is uniformly distributed over the n — 1 
bounding sphere, we first need to compute the ratio of such 
vectors x that form the angle (j> with x. 

But Yq is simply a vector of length sin(0) that lives in n — 2 
dimensional sphere of radius sin(0). Therefore, the probability 
density function of the pair forming an angle (p is the ratio of 
the volume of that n—2 dimensional sphere to the volume 
of the n — 1 dimensional sphere that bounds the space of all 
vectors. Denoting the volume of an n-dimensional unit sphere 
by Cn, the volume of a sphere of radius R is given by CnR", 
and its n — 1 dimensional surface by nC„i?"^^. Therefore, 
the ratio in question is 

C„_isin"-2(^) 



Given that 



P(0) = 



Or, 



lC/„ 



r"/2 



T[n/2 + 1] 

it is a simple exercise to verify the lemma. q.e.d. 

Now let us extend the lemma to the case of subspaces of 
dimension A: > 1. We assume that k random vectors have been 
chosen, which span a fc-dimensional subspace almost surely. 
We consider the case where a new vector is projected onto that 
fc-dimensional subspace (or onto its orthogonal complement). 
We have the following lemma: 

Lemma 4: The projection power of a random vector x onto 
the orthogonal complement of the space spanned by fc random 
vectors yi,-" ,yfc is given by 

2 n — k 



Proof: Again consider the vector x decomposed the into two 
orthogonal parts, Xp = xcos(0) and Xg = xsin((/)), where Xp 
lies in the space Sk spanned by yi, • • • ,yfc, and x^ lies in 



its complement. The angle (f) is the angle x forms with Sk.. 
Conditioning on 0, we compute the area on the surface of C„ 
where the endpoints of x can lie. The ratio of this area to 
the area of C„ will give us the probability of 0, since x is 
uniformly, and independently distributed over the n-sphere. 

The endpoints of Xp lie in St, in fact the surface of a fc- 
dimensional sphere of radius cos((/)). The differential volume 
of that sphere is fcCfc cos*''~^ ((/)). x^ is orthogonal to Sk and 
lies in the n — k — 1 dimensional sphere of radius sin(0) (one 
dimension is fixed because x has to lie on the surface of the 
n-sphere). The total differential area of possible endpoints for 
X is given by 

Area of endpoints of x with x- Sk = (p equals: 

kCk cos'=-i((/.)(n - fc)C„-fc sin"-^-i(0) 

The probability that x forms the angle with Sk is 

^ kCk cos'^-i(0)(n - k)Cn-k sin"-''^-i(0) 



Pr(x • Sk 



nCn 



This is illustrated in the figure below f or n = 3 and k — 2. 




The squared length of the projection Xp, i.e., the power of 
X in the subspace Sk is equal to cos^((/)), and therefore the 
average projected power of x onto Sk is 

.7r/2 

E||xp||2 = / Pr(x-5fc = 0)cos2(0)d0 
Jo 

r^/^ kCk cos''+\cf>){n - k)Cn^k sin"-^-i(0) 

7^ d(P 

nCn 



k/r 



(34) 



The last equation can be verified by any of the integration 



|xg|| , we obtain the 
q.e.d. 



tools. Complementing ||x|| = ||xp| 
lemma. 

As observed by the editor, equations ( |33] l and p4] i can also 
be derived using symmetry and isotropy without reference to 
the geometry used in this appendix. 
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